Non-Discounted Denumerable Markovian Decision Models

Non-discounted Denumerable Markovian Decision Models

Markov Decision Models with Weighted Discounted Criteria

We consider a discrete time Markov Decision Process with innnite horizon. The criterion to be maximized is the sum of a number of standard discounted rewards, each with a diierent discount factor. Situations in which such criteria arise include modeling investments, production, modeling projects of diierent durations and systems with multiple criteria, and some axiomatic formulations of multi-a...

On Non-denumerable Graphs

PROOF. We shall first prove that every complete graph of power t$i can be split up into the countable sum of trees. Let G be a complete graph of cardinal number ML Let {xa}, a<coi, be any well ordered set of power fc$i. We may assume that G is represented by a system of Segments (xa, Xp), a</3<coi. For any /3<coi arrange the set of all a< /3 into a sequence ap,n, n — \, 2, • • • , and let Gn be...

Denumerable Constrained Markov Decision Problems and Finite Approximations Denumerable Constrained Markov Decision Problems and Finite Approximations

The purpose of this paper is two fold. First to establish the Theory of discounted constrained Markov Decision Processes with a countable state and action spaces with general multi-chain structure. Second, to introduce nite approximation methods. We deene the occupation measures and obtain properties of the set of all achievable occupation measures under the diierent admissible policies. We est...

Denumerable State Nonhomogeneous Markov Decision Processes

We consider denumerable state nonhomogeneous Markov decision processes and extend results from both denumerable state homogeneous and finite state nonhomogeneous problems. We show that, under weak ergodicity, accumulation points of finite horizon optima (termed algorithmic optima) are average cost optimal. We also establish the existence of solution horizons. Finally, an algorithm is presented ...